Image processing device, image processing method and program

ABSTRACT

Embodiments of the technology involve apparatus and methods for control of displaying of images. In an example, an apparatus may include an image display, a sensor to detect posture of the image display and a processor to control sequentially displaying images of a group of images on the image display based on changes in the detected posture. The processor may control a display of a posture indicator on the image display such that the indicator may represent a relation between a change in the detected posture and an image of the group of images. Optionally, the indicator may be represented by a tilt meter. Moreover, in some embodiments, the sensor may be implemented with a gyroscopic sensor.

TECHNICAL FIELD

The present invention relates to an image processing device, and moreparticularly, to an image processing device and an image processingmethod for displaying an image, and a program for causing a computer toexecute the method.

BACKGROUND ART

In recent years, imaging devices such as a digital still camera or adigital video camera (e.g., a camera-integrated recorder) for imaging asubject such as a person or an animal to generate image data andrecording the image data as image content have been used.

For example, there is a reproduction apparatus for performing imageadvancing by user manipulation using a manipulation member andsequentially displaying a plurality of images. Also, there is areproduction apparatus in which a user changes a posture of thereproduction apparatus to change displayed content of a display unit.

For example, an information processing device for obtaining a movementamount or a rotation amount of a body and instructing, for example, toscroll displayed content of a display unit according to the amount hasbeen proposed (e.g., see Patent Literature 1).

CITATION LIST Patent Literature PTL 1

-   Japanese Patent Application Laid-Open No. H06-4208 (FIG. 3)

SUMMARY OF INVENTION

Some embodiments of the present technology may involve a method of aprocessor to control a display of images. The method may involvedetecting posture of an image display device with a sensor. The methodmay also involve sequentially displaying images of a group of images onthe image display device based on changes in the detected posture. Themethod may also involve displaying a posture indicator on the imagedisplay device, the indicator representing a relation between a changein the detected posture and an image of the group of images.

Optionally, the indicator may include a graphic tilt meter. In someembodiments, the processor may associate a normalized tilt angle with animage of the sequentially displayed images. The normalized tilt anglemay comprise an angle determined as a function of image capture angleinformation and a maximum tilt angle. In some embodiments, the imagecapture angle information may comprise a range of captured image angles.Still further, the maximum tilt angle may comprise a display viewingangle limit. In some cases, the method may also involve sequentiallydisplaying entrance images for groups of images on the image displaydevice based on changes in the detected posture. Optionally, the sensormay comprise a gyroscopic sensor.

Some embodiments of the present technology may involve an apparatus forcontrol of displaying of images. The apparatus may include an imagedisplay, a sensor to detect posture of the image display and a processorto control sequentially displaying images of a group of images on theimage display based on changes in the detected posture. The processormay control a display of a posture indicator on the image display suchthat the indicator may represent a relation between a change in thedetected posture and an image of the group of images.

Optionally, the indicator may comprise a tilt meter. Still further, theprocessor may control associating a normalized tilt angle with an imageof the sequentially displayed images. The detection of posture maycomprise a detection of angular velocity. Moreover, the processor maycontrol changing the display based on the angular velocity. In somecases of the apparatus, the processor may control calculating thenormalized tilt angle as a function of image capture angle informationand a maximum tilt angle. The image capture angle information maycomprise a range of captured image angles. The maximum tilt angle maycomprise a display viewing angle limit. In some such embodiments, theprocessor may control sequentially displaying entrance images for groupsof images on the image display device based on changes in the detectedposture. The sensor may include a gyroscopic sensor.

In still further embodiments of the technology may involve a system forcontrol of displaying images. The system may include a display means forimage displaying. The system may also include a posture means forsensing posture of the means for image displaying. The system may alsoinclude a control means for controlling sequentially displaying imagesof a group of images on the display means based on changes in the sensedposture. In some cases, the control means may control a display of ameans for posture indicating on the image display, the means for postureindicating representing a relation between a change in the sensedposture and an image of the group of images. Still further, the controlmeans may control associating of a normalized tilt angle with an imageof the sequentially displayed images. Moreover, the control means maycontrol calculating the normalized tilt angle as a function of imagecapture angle information and a maximum tilt angle. In some cases, suchan image capture angle information may comprise a range of capturedimage angles determined by the posture means.

TECHNICAL PROBLEM

According to the above-described related art, the displayed content ofthe display unit can be changed by changing the posture of a device,making it possible for a user to perform a changing manipulation in astate in which the user holds the device by hand.

Here, a case in which image advancing or image returning is performed ona plurality of images by changing posture of a device is assumed. Inthis case, a user performs a manipulation to change posture of thedevice while viewing images sequentially displayed on the display unit.Accordingly, it is understood that the user can easily perform themanipulation if he or she can easily recognize an image advancing orimage returning timing or a position relationship of display images inan image group while viewing images sequentially displayed on thedisplay unit.

In light of the foregoing, it is desirable to easily perform amanipulation to sequentially display a plurality of images using amanipulation method of changing posture of a device.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A shows an appearance configuration example of an imaging device100 in a first embodiment of the present invention.

FIG. 1B shows an appearance configuration example of an imaging device100 in a first embodiment of the present invention.

FIG. 2A shows one example of posture when the imaging device 100 is usedin the first embodiment of the present invention.

FIG. 2B shows one example of posture when the imaging device 100 is usedin the first embodiment of the present invention.

FIG. 2C shows one example of posture when the imaging device 100 is usedin the first embodiment of the present invention.

FIG. 3 is a block diagram showing a functional configuration example ofthe imaging device 100 in the first embodiment of the present invention.

FIG. 4 schematically shows stored content of the image managementinformation storage unit 210 in the first embodiment of the presentinvention.

FIG. 5A shows a display example of a representative image displayed onan input/output panel 101 by a representative image reproduction unit150 in the first embodiment of the present invention.

FIG. 5B shows a display example of a representative image displayed onan input/output panel 101 by a representative image reproduction unit150 in the first embodiment of the present invention.

FIG. 6 schematically shows an image display transition example in theinput/output panel 101 in the first embodiment of the present invention.

FIG. 7 schematically shows an image display transition example in theinput/output panel 101 in the first embodiment of the present invention.

FIG. 8A shows one example of a position relationship between a subjectand the imaging device 100 in an imaging operation performed using theimaging device 100, and an imaged image group generated in the imagingoperation in the first embodiment of the present invention.

FIG. 8B shows one example of a position relationship between a subjectand the imaging device 100 in an imaging operation performed using theimaging device 100, and an imaged image group generated in the imagingoperation in the first embodiment of the present invention.

FIG. 9A shows one example of a change of posture of the imaging device100 and angle information normalization performed by a manipulationmethod determination unit 180 in the first embodiment of the presentinvention.

FIG. 9B shows one example of a change of posture of the imaging device100 and angle information normalization performed by a manipulationmethod determination unit 180 in the first embodiment of the presentinvention.

FIG. 9C shows one example of a change of posture of the imaging device100 and angle information normalization performed by a manipulationmethod determination unit 180 in the first embodiment of the presentinvention.

FIG. 10 shows one example of a method of associating an image to bedisplayed and a tilt angle range in the manipulation methoddetermination unit 180, and an image displayed after the association andmanipulation support information in the first embodiment of the presentinvention.

FIG. 11 schematically shows a method of calculating a display area of anindicator in the manipulation method determination unit 180 in the firstembodiment of the present invention.

FIG. 12 shows a display example of an indicator based on a display areaof an indicator calculated by the manipulation method determination unit180 in the first embodiment of the present invention.

FIG. 13 schematically shows a method of determining the number of imagesto be displayed and a method of interleaving images to be displayed inthe manipulation method determination unit 180 in the first embodimentof the present invention.

FIG. 14 schematically shows a method of setting the display area of theindicator in the manipulation method determination unit 180 in the firstembodiment of the present invention.

FIG. 15 shows a display transition example of manipulation supportinformation (tilt meter) displayed by the group image reproduction unit160 in the first embodiment of the present invention.

FIG. 16 shows a display transition example of manipulation supportinformation (tilt meter) displayed by the group image reproduction unit160 in the first embodiment of the present invention.

FIG. 17A shows a display transition example of manipulation supportinformation (tilt meter) displayed by the group image reproduction unit160 in the first embodiment of the present invention.

FIG. 17B shows a display transition example of manipulation supportinformation (tilt meter) displayed by the group image reproduction unit160 in the first embodiment of the present invention.

FIG. 17C shows a display transition example of manipulation supportinformation (tilt meter) displayed by the group image reproduction unit160 in the first embodiment of the present invention.

FIG. 18 schematically shows a relationship between a display transitionin the input/output panel 101 and the posture of the imaging device 100in the first embodiment of the present invention.

FIG. 19 schematically shows a relationship between a display transitionin the input/output panel 101 and the posture of the imaging device 100in the first embodiment of the present invention.

FIG. 20 is a flowchart showing an example of a processing procedure inan image content reproduction process in the imaging device 100 in thefirst embodiment of the present invention.

FIG. 21 is a flowchart showing an example of a processing procedure inan image content reproduction process in the imaging device 100 in thefirst embodiment of the present invention.

FIG. 22 is a flowchart showing an example of a group image display modesetting process in a processing procedure of an image contentreproduction process in the imaging device 100 in the first embodimentof the present invention.

FIG. 23A shows manipulation support information (tilt meter) displayedby a group image reproduction unit 160.

FIG. 23B shows a display transition example in the input/output panel101 in a variant of the first embodiment of the present invention.

FIG. 24A shows manipulation support information (tilt meter) displayedby a group image reproduction unit 160.

FIG. 24B shows a display transition example in the input/output panel101 in a variant of the first embodiment of the present invention.

FIG. 25 shows a display transition example in the input/output panel 101in a variant of the first embodiment of the present invention.

FIG. 26 shows a display transition example in the input/output panel 101in a variant of the first embodiment of the present invention.

FIG. 27 shows a display transition example in the input/output panel 101in a variant of the first embodiment of the present invention.

FIG. 28 shows a display transition example in the input/output panel 101in a variant of the first embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

The description will be given in the following order.

1. First embodiment (display control: an example in which an indicatoror a tilt meter superimposed on an image in a group is displayed in agroup image display mode)

2. Variant 1. First Embodiment Appearance Configuration Example ofImaging Device

FIGS. 1A and 1B show an appearance configuration example of an imagingdevice 100 in a first embodiment of the present invention. FIG. 1A is aperspective view showing an appearance at a display surface side of theimaging device 100. FIG. 1B is a side view showing an appearance at thedisplay surface side of the imaging device 100. The imaging device 100is realized, for example, by a digital still camera or a digital videocamera (e.g., camera-integrated recorder) capable of imaging a subjectto generate image data, recording the image data as image content, anddisplaying the image content.

The imaging device 100 includes an input/output panel 101 and a shutterbutton 102. The imaging device 100 is an image processing device thatcan be carried by a user and the user can view an image displayed on theinput/output panel 101. The imaging device 100 is one example of animage processing device defined in the claims.

The input/output panel 101 displays various images and receives amanipulation input from a user by detecting a contact manipulation onthe input/output panel 101. The shutter button 102 is a button pressedby a photographer when image data (an imaged image) generated by imaginga subject is recorded as image content (a still image file).

The imaging device 100 includes other manipulation members such as apower switch and a mode switching switch, a lens unit, and the like, butsuch manipulation members will not be shown and described herein forease of explanation. Here, the lens unit (not shown) is an opticalsystem including a plurality of lenses for focusing light from thesubject, an aperture, and the like.

Here, a change of posture of the imaging device 100 will be described.For example, the user may change rotation angles (i.e., a yaw angle, apitch angle, and a roll angle) about three orthogonal axes in a state inwhich the user holds the imaging device 100 by hand. For example, theuser may change the posture of the imaging device 100 in a directionindicated by the arrow 302 (the yaw angle), the axis of which is thearrow 300 (or the arrow 301). The change example is shown in FIG. 2B.For another example, the user may change the posture of the imagingdevice 100 in a direction indicated by the arrow 305 (the pitch angle),the axis of which is the arrow 303 (or the arrow 304). For yet anotherexample, the user may change the posture of the imaging device 100 in adirection indicated by the arrow 308 (the roll angle), the axis of whichis the arrow 306 (or the arrow 307).

Further, for example, the user may change the posture of the imagingdevice 100 by moving (sliding) the imaging device 100 along a straightline on a plane in a state in which the user holds the imaging device100 by hand. For example, the user may change the posture by moving theimaging device 100 in the direction indicated by the arrow 300 or thearrow 301 (movement in a vertical direction in FIG. 1B). For anotherexample, the user may change the posture by moving the imaging device100 in a direction indicated by the arrow 303 or 304 (movement in ahorizontal direction in FIG. 1B). This change example is shown in FIG.2C. Further, for example, the user may change the posture by moving theimaging device 100 in a direction indicated by the arrow 306 or 307.

Use Example of Imaging Device

FIGS. 2A to 2C show one example of the posture when the imaging device100 is used in the first embodiment of the present invention. In FIG.2A, one example of the posture of the imaging device 100 when imagecontent reproduction is performed using the imaging device 100 is shown.When a person 310 performs the image content reproduction using theimaging device 100, the person 310 may view an image displayed on theinput/output panel 101, for example, in a state in which the person 310holds the imaging device 100 with both hands as shown in FIG. 2A.

In FIGS. 2B and 2C, a transition example in which the posture of theimaging device 100 is changed is shown. In FIG. 2B, a transition examplewhen the imaging device 100 is rotated in the direction indicated by thearrow 302 about the arrow 300 (301) shown in FIG. 1A to change theposture is shown.

In FIG. 2C, a transition example in which the imaging device 100 ismoved in the direction indicated by the arrows 303 and 304 shown inFIGS. 1A and 1B to change the posture is shown. In the first embodimentof the present invention, an example in which an image displayed on theinput/output panel 101 is sequentially changed by changing the postureof the imaging device 100 as such is shown. That is, an example in whichthe image displayed on the input/output panel 101 is sequentiallychanged by gesture manipulation by the user is shown. Such changes ofthe image display status will be described in detail with reference toFIGS. 6, 7, or the like.

Functional Configuration Example of Imaging Device

FIG. 3 is a block diagram showing a functional configuration example ofthe imaging device 100 in the first embodiment of the present invention.The imaging device 100 includes an input/output unit 110, a posturedetection unit 120, an analysis unit 130, a control unit 140, arepresentative image reproduction unit 150, a group image reproductionunit 160, a rendering memory 170, and the manipulation methoddetermination unit 180. Moreover, the imaging device 100 includes animage content storage unit 200, and an image management informationstorage unit 210.

The image content storage unit 200 stores image data (an imaged image)generated by the imaging unit (not shown) as an image file (imagecontent (still image content or moving image content)). The imagecontent storage unit 200 supplies the stored image content to therepresentative image reproduction unit 150 or the group imagereproduction unit 160. In the first embodiment of the present invention,an example in which the still image content is used as the image contentis shown.

The image management information storage unit 210 stores managementinformation (image management information) on the image content storedin the image content storage unit 200. Using the image managementinformation, reproduction in the representative image display mode andthe group image display mode is performed. Here, the representativeimage display mode is a mode in which a representative image of groupedimage contents and an image of non-grouped image content among the imagecontents stored in the image content storage unit 200 are sequentiallydisplayed according to user manipulation. The group image display modeis a mode in which images of each grouped image content among the imagecontents stored in the image content storage unit 200 are sequentiallydisplayed according to the change of the posture of the imaging device100. The image content storage unit 200 and the image managementinformation storage unit 210 may be, for example, one or a plurality ofremovable recording media, such as discs, such as digital versatilediscs (DVDs) or semiconductor memories such as memory cards. Therecording media may be embedded in the imaging device 100 or detachablyprovided in the imaging device 100.

The input/output unit 110 includes a display unit 111 and a manipulationreceiving unit 112. The display unit 111 is a display unit fordisplaying an image supplied from the representative image reproductionunit 150 or the group image reproduction unit 160. Various menu screensor various images are displayed on the display unit 111. The displayunit 111 may be, for example, a liquid crystal display (LCD) or anorganic electro luminescence (EL) panel.

The manipulation receiving unit 112 is a manipulation receiving unit forreceiving content of a manipulation by the user, and supplies amanipulation signal dependent on the received manipulation content tothe control unit 140. The manipulation receiving unit 112 correspondsto, for example, the manipulation member such as the shutter button 102shown in FIGS. 1A and 1B or a touch panel. For example, the touch panelis superimposed on the display unit 111 to transmit the screen of thedisplay unit 111, and receives a manipulation input from the user bydetecting an object contacting the display surface. That is, when a userpresses a desired location, for example, by touching the display surfacein the input/output unit 110 with his or her finger, the touch paneldetects a coordinate of the pressed location and outputs a manipulationsignal corresponding to the detected coordinate to the control unit 140.When the control unit 140 acquires the manipulation signal, the controlunit 140 executes a predetermined process based on the acquiredmanipulation signal. The manipulation receiving unit 112 may be, forexample, a touch panel for converting a contact on a display surfaceinto an electrical signal using a dedicated sensing device andoutputting the converted electrical signal to the control unit 140.Further, the manipulation receiving unit 112 may be, for example, atouch panel of an optical sensor type capable of detecting an object(e.g., a user's finger) approaching or contacting a display surface. Theinput/output unit 110 corresponds to a manipulation member such as theshutter button 102 or the input/output panel 101 shown in FIGS. 1A and1B.

The posture detection unit 120 detects a change of the posture of theimaging device 100 by detecting acceleration, movement, tilt and thelike of the imaging device 100, and outputs posture change informationabout the detected change of the posture to the analysis unit 130. Forexample, the posture detection unit 120 detects a movement direction anda movement amount in a specific direction of the imaging device 100, andan angular velocity in the specific direction (an angular velocity whenthe posture of the imaging device 100 is changed) as the change of theposture of the imaging device 100. The posture detection unit 120 may berealized by a gyro sensor (angular velocity sensor) or an accelerationsensor. The posture detection unit 120 is an example of a detection unitdefined in the claims.

The analysis unit 130 analyzes the change amount (e.g., the movementdirection and the movement amount) of the posture of the imaging device100 based on the posture change information output from the posturedetection unit 120 to generate analysis information for performingdisplay switching on the image or the manipulation support informationdisplayed on the display unit 111. The analysis unit 130 outputs theanalysis information to the group image reproduction unit 160.

The control unit 140 controls each unit of the imaging device 100 basedon the manipulation content from the manipulation receiving unit 112.For example, the control unit 140 sets the representative image displaymode when the manipulation receiving unit 112 receives a representativeimage display mode setting manipulation and sets the group image displaymode when the manipulation receiving unit 112 receives a group imagedisplay mode setting manipulation.

Further, when the group image display mode setting manipulation isreceived, the control unit 140 performs a control to sequentiallydisplay, on the display unit 111, the images in the group correspondingto the representative image displayed on the display unit 111 upon thesetting manipulation. When the group image display mode is set, thecontrol unit 140 performs a control to display, on the display unit 111,the manipulation support information indicating a relationship betweenthe change of the posture of the imaging device 100 and the displaystate in the display unit 111. The manipulation support information isinformation for supporting a manipulation to change the display state inthe display unit 111 and is, for example, an indicator 660 shown in FIG.18( a) or a tilt meter 670 shown in FIG. 19( a). Further, the controlunit 140 performs a control to display, on the display unit 111, themanipulation support information based on the position of the posture ofthe imaging device 100 associated by the manipulation methoddetermination unit 180 and the plurality of image contents.

When the representative image display mode has been set and aninstruction manipulation to instruct display switching (e.g., imageadvancing and image returning) of the image displayed on the displayunit 111 is received, the control unit 140 instructs the representativeimage reproduction unit 150 to switch the image display based on theinstruction manipulation. Further, when a group image display modesetting manipulation is performed, the control unit 140 acquires theimage management information (the manipulation method 215, or the like,shown in FIG. 4) associated with the group corresponding to therepresentative image displayed on the display unit 111 upon a settingmanipulation from the group image reproduction unit 160. The controlunit 140 is realized by, for example, a central processing unit (CPU).

The representative image reproduction unit 150 displays, on the displayunit 111, image content corresponding to the representative image andnon-grouped image content among the image contents stored in the imagecontent storage unit 200 under control of the control unit 140.Specifically, when the representative image display mode has been set,the representative image reproduction unit 150 acquires the imagecontent corresponding to the representative image and non-grouped imagecontent from the image content storage unit 200. Subsequently, therepresentative image reproduction unit 150 decodes the acquired imagecontent and renders a display image in the rendering memory 170 based onthe decoded image content. The representative image reproduction unit150 sequentially displays one of the images rendered in the renderingmemory 170 on the display unit 111 under control of the control unit140. The representative image reproduction unit 150 displays, on thedisplay unit 111, the representative image, and manipulation supportinformation (e.g., the manipulation support information 261 shown inFIG. 5) supporting the setting manipulation to the group image displaymode for displaying each image in the group corresponding to therepresentative image. Further, the representative image reproductionunit 150 displays, on the display unit 111, the representative image,and manipulation support information (e.g., the manipulation supportinformation 251, 252, 256, and 257 shown in FIG. 5) indicating therepresentative image, and a type of the group corresponding to therepresentative image (e.g., the group type 216 shown in FIG. 4).

The group image reproduction unit 160 displays the grouped imagecontents among the image contents stored in the image content storageunit 200 on the display unit 111 in a group unit under control of thecontrol unit 140. Specifically, when the group image display mode isset, the group image reproduction unit 160 acquires each image contentbelonging to the group corresponding to the representative imagedisplayed upon the group image display mode setting manipulation fromthe image content storage unit 200. Further, the group imagereproduction unit 160 acquires the image management informationassociated with each image content belonging to the group from the imagemanagement information storage unit 210. Subsequently, the group imagereproduction unit 160 outputs the acquired image management informationto the manipulation method determination unit 180. Moreover, forexample, the group image reproduction unit 160 decodes the acquiredimage content and renders a display image in the rendering memory 170based on the decoded image content. In this case, the group imagereproduction unit 160 arranges display images and renders the images inthe rendering memory 170, for example, according to a predeterminedrule.

When the group image display mode setting manipulation is received, thegroup image reproduction unit 160 displays each image belonging to agroup to be displayed and manipulation support information on thedisplay unit 111 to be associated with each other. In this case, forexample, the group image reproduction unit 160 changes content of themanipulation support information based on the change of the posture ofthe imaging device 100 detected by the posture detection unit 120. Thatis, the group image reproduction unit 160 sequentially displays, on thedisplay unit 111, one image from among the images rendered in therendering memory 170 based on the image display switching instructionoutput from the analysis unit 130 (an image advancing or returninginstruction).

The rendering memory 170 is a rendering buffer for holding the imagesrendered by the representative image reproduction unit 150 or the groupimage reproduction unit 160, and supplies the rendered images to therepresentative image reproduction unit 150 or the group imagereproduction unit 160.

The manipulation method determination unit 180 determines themanipulation method when grouped image content is in a group unitdisplayed on the display unit 111, based on the image managementinformation output from the group image reproduction unit 160. Forexample, the manipulation method determination unit 180 normalizes theangle information assigned to a plurality of image contents, for eachimage content, using a range in which the posture of the imaging device100 is changed, as a reference. The manipulation method determinationunit 180 determines the manipulation method by associating the positionof the posture of the imaging device 100 with the plurality of imagecontents, for each image content, based on the normalized angleinformation, and outputs the determined manipulation method to the groupimage reproduction unit 160. A method of determining the manipulationmethod will be described in detail with reference to FIGS. 8A to 14.

(Stored Content of Image Management Information Storage Unit)

FIG. 4 schematically shows stored content of the image managementinformation storage unit 210 in the first embodiment of the presentinvention. Group identification information 211, image contentidentification information 212, group presence 213, a representativeimage 214, a manipulation method 215, and a group type 216 are stored inthe image management information storage unit 210.

Identification information for identifying each group is stored in thegroup identification information 211. Here, the group is an image groupincluding a plurality of image contents associated with one another, theorder of which is specified based on a predetermined rule. In the firstembodiment of the present invention, an example in which an image groupincluding a plurality of image contents, and image contents not in theimage group are assigned group identification information to identifythe image contents not in the image group and the image group is shown.

Identification information (image content identification information)for identifying each image content stored in the image content storageunit 200 is stored in the image content identification information 212.For example, the image content identification information of each imagecontent is stored in an image advancing or returning order (e.g., anorder of photographing time (record date)).

Angle information corresponding to an imaging operation when the imagecontent stored in the image content storage unit 200 is generated isstored in the angle information 213. For example, when a multi-viewimage is photographed using the imaging device 100 and the multi-viewimage generated by the photograph is stored in the image content storageunit 200, an angle upon photographing (an angle when a predeterminedposition is used as a reference) is stored as the angle information.“Absence” is shown in the field of the image content not assigned theangle information. Here, the multi-view image is a plurality of imagesgenerated by an imaging operation changing a view with respect to atarget subject (e.g., a face of a person) by moving the imaging device100 using the target subject as a reference. The imaging operation willbe described in detail with reference to FIG. 8A.

Representative image information for specifying a representative imageamong a plurality of grouped image contents is stored in therepresentative image 214. In the example shown in FIG. 4, image contentidentification information (image content identification number) ofimage content corresponding to the representative image is shown as therepresentative image information. Further, “absence” is indicated in thefield of the non-grouped image content. That is, it is possible toidentify whether the image contents are grouped (whether the imagecontents belong to a group) based on whether the image contentidentification information is stored in the representative image 214.

A manipulation method when a plurality of grouped image contents aresequentially displayed in the case in which the group image display modehas been set is stored in the manipulation method 215. The manipulationmethod is a manipulation method relating to the change of the posture ofthe imaging device 100. For example, “horizontal rotation” refers to amanipulation to sequentially display a plurality of image contents byrotating the imaging device 100 in the direction indicated by the arrow302 about the arrow 300 (301) shown in FIGS. 1A and 1B to change theposture. Further, for example, “vertical rotation” refers to amanipulation to sequentially display a plurality of image contents byrotating the imaging device 100 in the direction indicated by the arrow305 about the arrow 303 (304) shown in FIGS. 1A and 1B to change theposture. Further, for example, “horizontal movement” refers to amanipulation to sequentially display a plurality of image contents bymoving the imaging device 100 in the direction indicated by the arrows303 and 304 shown in FIGS. 1A and 1B to change the posture. For example,a “vertical movement” refers to a manipulation to sequentially display aplurality of image contents by moving the imaging device 100 in thedirection indicated by the arrows 300 and 301 shown in FIGS. 1A and 1Bto change the posture. The imaging device 100 can automaticallydetermine the manipulation method, for example, according to an imagingoperation method upon generating the image contents. Further, themanipulation method may be appropriately determined according to theuser's preference by the user manipulation.

The type relating to a group (image group) including a plurality ofimage contents the order of which is specified based on a predeterminedrule and that are associated with each other is stored in the group type216. In FIG. 4, an example in which “multi-shot image” and “multi-viewimage” are stored as the group type is shown. Here, “multi-shot image”is the type of a group including a plurality of images generated bymulti-shot photographing and “multi-view image” is the type of a groupincluding a plurality of images generated by multi-view photographing.“Absence” is indicated in the field of the non-grouped image content.Further, when the representative image display mode has been set, therepresentative image and the type of a group corresponding to therepresentative image based on content of the group type 216 can bedisplayed. This display example is shown in FIG. 5.

The information stored in the group identification information 211, thegroup presence 213, the representative image 214, the manipulationmethod 215 and the group type 216 may be recorded in an image file(image content). For example, such information can be recorded asattribute information relating to a still image file. The informationrecorded in the image content may be used upon reproduction of the imagecontent.

Display Example of Representative Image

FIGS. 5A and 5B shows a display example of the representative imagedisplayed on the input/output panel 101 by the representative imagereproduction unit 150 in the first embodiment of the present invention.In FIG. 5A, a display example when a representative image of a group inwhich “multi-view image” is stored in the group type 216 is displayed isshown. Further, in FIG. 5B, a display example in which a representativeimage of a group in which “multi-shot image” is stored in the group type216 is displayed is shown.

As shown in FIG. 5A, when the representative image 250 of the grouprelating to the multi-view image is displayed on the input/output panel101, manipulation support information (decoration image) 251 and 252indicating that the group is a group relating to the multi-view image isdisplayed at both sides of the representative image 250. Further, agroup type display area 262 and an image size display area 263 areprovided on the representative image 250, and manipulation supportinformation 261 for setting the group image display mode is providedbelow the representative image 250.

In the group type display area 262, the type of the group correspondingto the displayed representative image is displayed. For example, whenthe type of the group corresponding to the displayed representativeimage is a group relating to the multi-view image, “multi-view” isdisplayed in the group type display area 262.

In the image size display area 263, an image size of each imagebelonging to the group corresponding to the displayed representativeimage (e.g., an image size of one image) is displayed.

The manipulation support information 261 is a manipulation button forsupporting a manipulation to set the group image display mode uponsetting of the representative image display mode. That is, the user canset the group image display mode by pressing the manipulation supportinformation 261. Further, the same letters as the group type in thegroup type display area 262 is displayed in a rectangle corresponding tothe manipulation support information 261.

Further, as shown in FIG. 5B, when a representative image 255 of a grouprelating to a multi-shot image is displayed on the input/output panel101, manipulation support information (decoration image) 256 and 257indicating that the group is the group relating to the multi-shot imageis displayed at both sides of the representative image 255. Further, agroup type display area 262 and an image size display area 263 areprovided on the representative image 255, and manipulation supportinformation 261 for setting the group image display mode is providedbelow the representative image 255.

Thus, when the representative image display mode has been set, a displayaspect of the representative image is changed for each image group basedon correlativity (e.g., corresponding to group type) between images inthe image group associated with the representative image displayed onthe input/output panel 101, and then the representative image isdisplayed.

As shown in FIG. 5B, when the type of the group corresponding to thedisplayed representative image is the group relating to the multi-shotimage, “multi-shot” is displayed in the group type display area 262.Further, “multi-shot” is displayed in a rectangle corresponding to themanipulation support information 261.

Image Display Transition Example

FIGS. 6 and 7 schematically show image display transition examples onthe input/output panel 101 in the first embodiment of the presentinvention. In FIG. 6( a), a case in which the image contents stored inthe image content storage unit 200 are arranged according to apredetermined rule (order of image content identification information)is schematically shown. In the example shown in FIG. 6( a), only imagecontents #1 to #13 among the image contents stored in the image contentstorage unit 200 are shown. Further, the image contents #1 to #13correspond to #1 to #13 indicated in the image content identificationinformation 212 of FIG. 4. In FIG. 6, image contents and correspondingimages are schematically shown by rectangles with image contentidentification information (#1 to #13). Further, grouped image contentsare surrounded in a group unit by dotted rectangles.

In FIG. 6( b), a transition of images displayed on the input/outputpanel 101 when the representative image display mode has been set isshown. As shown in FIG. 6( b), non-grouped image contents (#1, #5, #6,and #13) and representative images (#3 and #9) of the grouped imagecontents are sequentially displayed based on an image advancingmanipulation or an image returning manipulation by the user. The imageadvancing manipulation or the image returning manipulation is performed,for example, using the manipulation member included in the imagingdevice 100 (e.g., the touch panel or an external manipulation button(not shown)). Further, the image advancing manipulation or the imagereturning manipulation may be performed, for example, according to thechange of the posture of the imaging device 100, similar to setting ofthe group image display mode.

In FIG. 7( a), a display transition example in which image contents #6,#9 and #13 among the images shown in FIG. 6( b) are sequentiallydisplayed based on the image advancing manipulation or the imagereturning manipulation by the user is shown.

In FIG. 7, images indicating image contents using rectangles with imagecontent identification information (#6, #8 to #10, and #13) areschematically shown. Thus, it is possible to display each image on theinput/output panel 101 based on the user manipulation. Here, when therepresentative image (e.g., #9) of the grouped image content isdisplayed on the input/output panel 101 as shown in FIG. 7( a),manipulation support information 261 for setting the group image displaymode is displayed. This manipulation support information 261 is the sameas the manipulation support information 261 shown in FIG. 5. That is,the user can set the group image display mode by pressing a rectangulararea corresponding to the manipulation support information 261. Forexample, the group image display mode is set when the manipulationsupport information 261 is pressed in a state in which representativeimages (e.g., #3 and #9) of the grouped image contents are displayed onthe input/output panel 101. Thus, the manipulation to press themanipulation support information 261 becomes an entrance to the groupimage display mode. Where the group image display mode is set, theimages contents belonging to the group corresponding to therepresentative image displayed when the group image display mode settingmanipulation (a manipulation to press the manipulation supportinformation 401) is performed are sequentially displayed.

On the other hand, when the non-grouped image contents (e.g., #6 and#13) are displayed on the input/output panel 101 as shown in FIG. 7( a),the manipulation support information 261 is not displayed. Accordingly,the manipulation to press the manipulation support information 261 willnot be performed and the group image display mode will not be set in astate in which the non-grouped image contents (e.g., #6 and #13) havebeen displayed on the input/output panel 101.

Here, a case in which a group image display mode setting manipulation(the manipulation to press the manipulation support information 261) isperformed in a state in which the representative image (#9) of thegrouped image content has been displayed on the input/output panel 101is assumed. In this case, the group image reproduction unit 160 acquiresimage contents belonging to the group corresponding to therepresentative image displayed when the group image display mode settingmanipulation is performed, from the image content storage unit 200.Further, the group image reproduction unit 160 acquires image managementinformation associated with each image content belonging to the groupfrom the image management information storage unit 210. Subsequently,the manipulation method determination unit 180 determines an image thatis an object to be displayed (an image to be displayed) selecting fromeach image content belonging to the group based on the acquired imagemanagement information as well as determining a manipulation method fordisplaying these images to be displayed. The method of determining thisimages to be displayed and the method of determining the manipulationmethod thereof will be described in detail with reference to FIGS. 8A to14.

Subsequently, the group image reproduction unit 160 renders each imagein the rendering memory 170 based on the acquired image content. In thiscase, the group image reproduction unit 160 renders each image in therendering memory 170 according a predetermined rule (e.g., the order ofimage content identification information). An image rendering example inthe rendering memory 170 is shown in FIG. 6( c).

In FIG. 6( c), a case in which the image contents belonging to the groupcorresponding to the representative image displayed when the group imagedisplay mode is set are rendered in the rendering memory 170 isschematically shown. When an image content rendering process is thusperformed, the group image reproduction unit 160 displays the fact onthe input/output panel 101 and does not perform a displaying process inthe group image display mode. A display example in this case is shown inFIG. 7( b). Note that FIG. 6( b) shows an image rendering example whenall of each image content belonging to a group #105 has been determinedas the image to be displayed.

In FIG. 7( b), a display example when the image content renderingprocess is performed directly after the group image display mode settingmanipulation is performed is shown. For example, manipulation supportinformation 402 indicating the image content rendering processsuperimposed on the representative image (#9) displayed when the groupimage display mode is set is displayed on the input/output panel 101.When the image content rendering process is terminated, the group imagereproduction unit 160 displays manipulation support information forsupporting the image display switching manipulation in the group imagedisplay mode on the input/output panel 101. The display example is shownin FIG. 7( c).

In FIG. 7( c), a display example when the image content renderingprocess is terminated after the group image display mode settingmanipulation is performed is shown. For example, manipulation supportinformation 403 and 404 for supporting the image display switchingmanipulation in the group image display mode superimposed on therepresentative image (#9) displayed when the group image display mode isset is displayed on the input/output panel 101. The manipulation supportinformation 403 and 404 is manipulation guides for supporting amanipulation for displaying each image in the group displayed on theinput/output panel 101. The manipulation support information 404 shownin FIG. 7( c) is for supporting manipulation when the user performs amanipulation to rotate the imaging device 100 about the arrow 300 (301)shown in FIG. 1A in a state in which the user holds the imaging device100 in his or her hand. Further, the manipulation support information403 is manipulation support information to be displayed at first, whichcan be deleted by the user's operation or after a certain period timehas elapsed. That is, when it is recognized that the user understandsthe meaning of the message, the message will be deleted. By deleting inthis way, it is possible to notify the user that a manipulation to tiltis a right manipulation. Note that each of figures shows each ofmanipulation support information relatively large, for ease ofexplanation.

Here, as shown in FIG. 7( c), a setting manipulation to set therepresentative image display mode in a state in which the group imagedisplay mode is set and the manipulation support information 403 and 404superimposed on the image to be displayed is displayed will bedescribed. When the representative image display mode is set in thisstate, for example, the user performs a touch manipulation to contactthe display surface of the input/output panel 101. When the touchmanipulation is performed, the mode is switched from the group imagedisplay mode to the representative image display mode (i.e., therepresentative image display mode is set), as shown by the arrow 405.Directly after the representative image display mode is set as such, arepresentative image (e.g., image #9) of the group displayed uponsetting of the group image display mode is displayed. The representativeimage display mode may be set by a setting manipulation using themanipulation member, as well as the touch manipulation in theinput/output panel 101.

Assignment Example of Angle Information

FIGS. 8A and 8B shows one example of a position relationship between thesubject and the imaging device 100 in an imaging operation performedusing the imaging device 100 and an imaged image group generated in theimaging operation in the first embodiment of the present invention.

In FIG. 8A, a position relationship between the target subject (the faceof the person 500) and the imaging device 100 in the imaging operationperformed using the imaging device 100 is schematically shown. In theexample shown in FIG. 8A, an example in which the face of the person 500is a target subject and the imaging operation is performed by moving theimaging device 100 to change a view with respect to the face of theperson 500 is shown. Specifically, in FIG. 8A, a case in which the frontof the person 500 is photographed (when the angle information Ai in theimaging operation is 0 degree) is shown. Similarly, in FIG. 8A, a casein which a right oblique side of the person 500 is photographed (theangle information Ai in the imaging operation is −60 degrees) and a casein which a left oblique side of the person 500 is photographed (theangle information Ai in the imaging operation is +60 degrees) are shown.In FIG. 8A, for example, a face relating to the person 500 (includingthe person 500 included in each image) is shown in a simplified form.Further, the angle information Ai in the imaging operation performedusing the imaging device 100 is shown below the imaging devices 100(Ai=−60 degrees, Ai=0 degree and Ai=+60 degrees).

In FIG. 8B, an imaged image group (the images (C0 to C5) 510 to 515)generated by the imaging device 100 is shown. Ci is identificationinformation for identifying each image, and i is an integer satisfying0<=i<=n. In the example shown in FIG. 8B, n=5.

Specifically, the images (C0 to C2) 510 to 512 are imaged images whenthe right oblique side of the person 500 is photographed (when the angleinformation Ai in the imaging operation is −60 degrees, −55 degrees, and−25 degrees). Further, the images (C3 to C5) 513 to 515 are imagedimages when the left oblique side of the person 500 is photographed(when the angle information Ai in the imaging operation is +40 degrees,+55 degrees, and +60 degrees). In FIG. 8B, for example, the facerelating to the person 500 included in the images (C0 to C5) 510 to 515is shown in a simplified form, similar to FIG. 8A. Further, the angleinformation Ai in the imaging operation (an angle in the imagingoperation of the image Ci) is indicated below each image (A0=−60degrees, A1=−55 degrees, . . . , A4=+55 degrees, A5=+60 degrees).

Thus, the images (C0 to C5) 510 to 515 generated by changing a view withrespect to the target subject (the face of the person 500) are groupedas a multi-view image and stored in the image content storage unit 200.Further, such image management information (e.g., the angle informationand the group type) is stored in the image management informationstorage unit 210. When the images (C0 to C5) 510 to 515 are displayedusing the imaging device 100, the images (C0 to C5) 510 to 515 in thegroup can be sequentially displayed by horizontally rotating the imagingdevice 100.

Here, the images (C0 to C5) 510 to 515 are images obtained by imaging aface of the same person from different views. Accordingly, the user canobtain the feeling of viewing a three dimensional object (the face ofthe person 500) by rapidly performing the user manipulation to tilt theimaging device 100 to the left and right and rapidly performing imageadvancing or image returning. That is, the user manipulation to tilt theimaging device 100 to the left and right gives the user the feeling ofviewing a pseudo-three-dimensional image.

Thus, it is possible to enjoy a pseudo-three-dimensional image as aprofound image by performing high-speed image advancing or imagereturning by a gesture manipulation of the user on the image groupobtained by photographing the target subject from several angles. Thatis, when the user desires to three-dimensionally view the object, suchas a face of a person or a vehicle, he or she can easily view apseudo-three-dimensional image of the object through a manualmanipulation.

When the image group is displayed as such, a determination of the angleinformation (the overall rotation angle range and the reference angle)associated with the manipulation method is important. Hereinafter, amethod of determining the angle information associated with themanipulation method will be described in detail with reference to thedrawings.

Normalization Example of Angle Information

FIGS. 9A to 9C shows one example of a change of the posture of theimaging device 100 and normalization of the angle information performedby the manipulation method determination unit 180 in the firstembodiment of the present invention. In FIG. 9A, a transition example ofthe imaging device 100 in the case in which the image contents stored inthe image content storage unit 200 are displayed when the group imagedisplay mode has been set is shown. In the example shown in FIG. 9A, anappearance at a bottom side of the imaging device 100 (i.e., a surfaceopposite to a surface on which the shutter button 102 is provided) isshown. Similarly, in FIGS. 9B, 10(b) and 13(a), the appearance at thebottom side of the imaging device 100 is shown.

In FIG. 9B, a manipulation range (the overall range of the rotationangle) of the imaging device 100 corresponding to the transition of theimaging device 100 is schematically shown. In this example, an examplein which image advancing or image returning is performed by rotating theimaging device 100 about the arrow 300 (301) shown in FIGS. 1A and 1B tochange the posture is shown. In this case, the tilt angle (referenceangle) as a reference when image advancing or image returning isperformed is Vi. The tilt angle Vi may be obtained by Equation 1:

Vi=(gamma)*j  Equation 1

Here, (gamma) is a value indicating tilt accuracy that can be maintainedby a person, for example, without hand shaking. Such a value may bedetermined in consideration of accuracy or performance of the posturedetection unit 120 (a tilt detection device). Further, j is a valuesatisfying (integer of −m<=j<=m), and i is an integer satisfying0<=i<=n. In the example shown in FIG. 9B, n=4.

Here, the angle information Ai assigned to each image Ci in the imagegroup is an angle with respect to the target subject in an imagingoperation. Accordingly, it is preferable to associate each image Ci withthe tilt angle Vi based on the angle information Ai. The angleinformation Ai is normalized so that the angle information Aicorresponds to the tilt angle Vi. Specifically, the normalization angleinformation Bi for which the angle information Ai has been normalizedmay be obtained by Equation 2:

Bi=(Vmax/Aabs)*Ai  Equation 2

Here, Vmax is an absolute value of a maximum tilt angle and is an angleat which a user is allowed to view the display screen. Vmax is limitedto, for example, a viewing angle of the screen. Further, Aabs is a value(absolute value) indicating a range of the angle information Ai.

In the example shown in FIG. 9B, an example in which Vmax=180 degrees,(gamma)=45 degrees, and m=2 (i.e., −2<=j<=2) is shown. Further, therange of the angle information Ai is specified by the angle informationA0(=−60 degrees) of the image (C0) 510 and the angle information A5(=+60degrees) of the image (C5) 515. That is, Aabs=|−60 degrees−60degrees|=120 degrees.

Accordingly, in the example shown in FIG. 9B, B0=(180 degrees (Vmax)/120degrees (Aabs))*(−60 degrees (A0))=−90 degrees. Similarly, B1=−82.5degrees, B2=−37.5 degrees, B3=60 degrees, B4=82.5 degrees, and B5=90degrees. Such normalization angle information Bi is calculated by themanipulation method determination unit 180.

In FIG. 9C, the images (C0 to C5) 510 to 515, angle information Aiassigned to the images and normalization angle information Bi calculatedfor the images are associated with one another and shown.

The image Ci can be properly assigned to the tilt angle (referenceangle) Vi by normalizing the angle information in the imaging operationassociated with the image Ci.

Association Example Between Images to be Displayed and Tilt Angle

FIG. 10 shows one example of a method of associating images to bedisplayed and the tilt angle range in the manipulation methoddetermination unit 180, and the images displayed after the associationand manipulation support information in the first embodiment of thepresent invention. In FIGS. 10( a) and (b), the method of associatingthe images to be displayed and the tilt angle range in the manipulationmethod determination unit 180 is schematically shown.

In FIG. 10( a), images (C0 to C5) 510 to 515, angle information Aiassigned to the images, and normalization angle information Bicalculated for the images are associated with one another and shown. Theexample shown in FIG. 10( a) is the same as that shown in FIG. 9C exceptthat a contour corresponding to a rectangle and a person 500corresponding to the image (C1) 511 is indicated by a dotted line.

In FIG. 10( b), a manipulation range of the imaging device 100 (anoverall range of the rotation angle) corresponding to a transition ofthe imaging device 100 is schematically shown. The example shown in FIG.10( b) is the same as that shown in FIG. 9B.

Here, in the example shown in FIGS. 10( a) and (b), the number of theimages (C0 to C5) 510 to 515 (6) is greater than the number of tiltangles (reference angles) Vi (5). When the number of images belonging tothe group to be displayed is greater than the number of tilt angles(reference angles) Vi, an interleaving process is performed on theimages belonging to the group to be displayed. That is, the manipulationmethod determination unit 180 extracts a predetermined number of imagesfrom among the images belonging to the group to be displayed, as imagesto be displayed, so that the number of the images is in the range of thenumber of tilt angles (reference angles) Vi.

Specifically, the image Ci associated with the normalization angleinformation Bi closest to the tilt angle (reference angle) Vi isselected as an image CCi for each tilt angle (reference angle) Vi fromthe image group (image Ci (0<=i<=n)) for which the normalization angleinformation Bi has been calculated.

In FIG. 10, an example in which the image Ci associated with thenormalization angle information Bi closest to the tilt angle (referenceangle) Vi from the tilt angle (reference angle) V0 to V4 is selected,and the selected image Ci and the tilt angle (reference angle) Vi areassociated with each other is shown.

Specifically, the image C0 associated with the normalization angleinformation B0(=−90 degrees) closest to the tilt angle V0(=−90 degrees)is selected and associated with the tilt angle V0, as an image CC0.Similarly, the image C2 associated with the normalization angleinformation B2(=−37.5 degrees) closest to the tilt angle V1(=−45degrees) is selected and associated with the tilt angle V1, as an imageCC1. Further, the image C3 associated with the normalization angleinformation B3(=+60 degrees) closest to the tilt angle V2(=0 degree) isselected and associated with the tilt angle V2, as an image CC2.Further, the image C4 associated with the normalization angleinformation B4(=+82.5 degrees) closest to the tilt angle V3(=45 degrees)is selected and associated with the tilt angle V3, as an image CC3.Further, the image C5 associated with the normalization angleinformation B5(=+90 degrees) closest to the tilt angle V4(=90 degrees)is selected and associated with the tilt angle V4 as an image CC4.

That is, CCi(={C0, C2, C3, C4, C5}={CC0, CC1, CC2, CC3, CC4}) isselected from the image group (image Ci (0<=i<=5)).

The selected images CCi associated with the tilt angles Vi aresequentially displayed according to the tilt of the imaging device 100.For example, when the posture of the imaging device 100 is in a state ofthe tilt angle (reference angle) Vi, the image CCi associated with thetilt angle (reference angle) V is displayed. The image CCi continues tobe displayed until the posture of the imaging device 100 becomes thetilt angle (reference angle) Vi−1 or the tilt angle (reference angle)Vi+1.

In FIG. 10( c), a display example when each image CCi associated withthe tilt angle Vi by the association method shown in FIG. 10 aredisplayed on the input/output panel 101 is shown. In the example shownin FIG. 10( c), an example in which manipulation support information 520superimposed on the image to be displayed (the image CCi associated withthe tilt angle Vi) is displayed is shown.

The manipulation support information 520 is an indicator indicating arelationship between the images belonging to the group to be displayed(all extracted images to be displayed) and images displayed on theinput/output panel 101 when the group image display mode has been set.Specifically, the manipulation support information 520 indicates therelationship between all the extracted images to be displayed (C0 and C2to C5) 510, and 512 to 515 and one image displayed on the input/outputpanel 101. For example, a rectangular area in the manipulation supportinformation 520 corresponding to one image displayed on the input/outputpanel 101 among all the images (C0 and C2 to C5) 510 and 512 to 515 hasa different display aspect from other areas. In the example shown inFIG. 10( c), corresponding symbols (V0 to V4) are added to rectanglesindicating the manipulation support information 520, for ease ofexplanation.

That is, in the state shown in FIG. 10( c), a state in which the image(C3) 513 among the images to be displayed (C0 and C2 to C5) 510 and 512to 515 is displayed on the input/output panel 101 is shown. Further,when the imaging device 100 is rotated to the left and exceeds V1 shownin FIG. 10( b), the rectangular area V1 in the manipulation supportinformation 520 has a different display aspect from the other areas.Further, when the imaging device 100 is rotated to the left and exceedsV0 shown in FIG. 10( b), the rectangular area V0 in the manipulationsupport information 520 has a different display aspect from the otherareas. On the other hand, when the imaging device 100 is rotated to theright and exceeds V3 and V4 shown in FIG. 10( b), the rectangular areasV3 and V4 in the manipulation support information 520 similarly havedifferent display aspects from the other areas.

Thus, when the group image display mode has been set, the total numberof images to be displayed and a position of a display image in all theimages to be displayed (a position in a display order) can be easilyrecognized by displaying both the image to be displayed and theindicator.

In the foregoing, the example in which the angle information assigned toeach image is normalized, the images are assigned to the angles atcertain intervals, and the images are displayed according to the angleshas been shown. Here, the tilt angle (reference angle) may be determinedaccording to the interval of the calculated normalization angleinformation of each image and each image may be displayed according tothe angle. Further, when the indicator is displayed as the manipulationsupport information, the display area of the indicator may be calculatedaccording to the interval of the calculated normalization angleinformation of each image. A method of calculating the display area ofthe indicator and a display example are shown in FIGS. 11 and 12.

(Variant of Indicator Display)

FIG. 11 schematically shows a method of calculating the display area ofthe indicator in the manipulation method determination unit 180 in thefirst embodiment of the present invention.

In FIG. 11( a), images (C0 to C5) 510 to 515, angle information Aiassigned to the images, and normalization angle information Bicalculated for the images are associated with one another and shown.Since the example shown in FIG. 11( a) is the same as that shown in FIG.10( a), a description thereof will be omitted herein.

In FIG. 11( b), a relationship between the calculated display area andthe indicator is schematically shown. For example, the manipulationmethod determination unit 180 performs normalization on thenormalization angle information Bi calculated for the images (C0 to C5)510 to 515. Specifically, the manipulation method determination unit 180calculates the normalization angle information BBi(=normalization angleinformation of CCi+90 degrees). The display area Wi of the indicator maybe obtained by Equation 3 using the normalization angle information BBi.

Wi=BBi/Vmax*L  Equation 3

Here, L is a value indicating a length of the indicator displayed on theinput/output panel 101. Further, BBi is a value calculated by theabove-described normalization.

Here, the display area Wi of the indicator is a value indicating alength from a left end in the indicator (a length L) shown in FIG. 11(b). A display example based on the calculated display area Wi of theindicator is shown in FIG. 12.

FIG. 12 shows a display example of the indicator based on the displayarea of the indicator calculated by the manipulation methoddetermination unit 180 in the first embodiment of the present invention.

In FIG. 12( a), the relationship between the display area Wi calculatedin the example shown in FIG. 11 and the indicator is schematicallyshown. The example shown in FIG. 12( a) is the same as that shown inFIG. 11( b) except that symbols (C0 and C2 to C5) of images are added torectangles corresponding to positions on the indicator specified by thedisplay area Wi calculated in the example shown in FIG. 11.

In FIG. 12( b), a state in which an indicator (manipulation supportinformation) 530 corresponding to the relationship between the displayarea Wi and the indicator shown in FIG. 12( a) is displayed on theinput/output panel 101 is shown. In the example shown in FIG. 12( b), astate in which an area corresponding to the image (C3) 513 has adifferent display aspect from other areas is shown.

In FIG. 12( c) to (f), a display state on the input/output panel 101 ofthe indicator 530 corresponding to the relationship between the displayarea Wi and the indicator shown in FIG. 12( a) is shown. In the exampleshown in FIG. 12( c), a state in which an area corresponding to theimage (C2) 512 has a different display aspect from the other areas isshown, and in the example shown in FIG. 12( d), a state in which an areacorresponding to the image (C0) 510 has a different display aspect fromthe other areas is shown. Further, in the example shown in FIG. 12( e),a state in which an area corresponding to the image (C4) 514 has adifferent display aspect from the other areas is shown, and in theexample shown in FIG. 12( f), a state in which an area corresponding tothe image (C5) 515 has a different display aspect from the other areasis shown.

In the above description, the case in which image contents stored to beassociated with the angle information are displayed has been describedby way of example. However, a case in which image contents in whichangle information is not recorded in an imaging operation are displayedis assumed. Hereinafter, the case in which the image contents in whichangle information is not recorded in the imaging operation is displayedwill be described.

Determination Example of the Number of Images to be Displayed andInterleaving Example of Images to be Displayed

FIG. 13 schematically shows a method of determining the number of imagesto be displayed and a method of interleaving the images to be displayedin the manipulation method determination unit 180 in the firstembodiment of the present invention.

Here, if the number of images to be displayed is great in the case inwhich image content in which angle information is not recorded in animaging operation is displayed, image advancing or image returning islikely to be performed even when a tilt manipulation of the imagingdevice 100 by a user is small. In this case, unintentional imageadvancing or image returning is assumed to be performed. In the case inwhich image content in which the angle information is not recorded inthe imaging operation is displayed, the number of images to be displayedis determined according to a range in which the tilt manipulation of theimaging device 100 can be performed by the user, and only the images tobe displayed are displayed.

In FIG. 13( a), a manipulation range (overall rotation angle range) ofthe imaging device 100 corresponding to the transition of the imagingdevice 100 is schematically shown. The manipulation range of the imagingdevice 100 shown in FIG. 13( a) is substantially the same as that shownin FIG. 9B, for example.

For example, when the manipulation method determination unit 180determines the number M of images to be displayed, the manipulationmethod determination unit 180 may obtain the number M using Equation 4.

M=Vmax/(gamma)+1  Equation 4

Here, M is a value indicating the number of images that can bedisplayed. Further, Vmax is an absolute value of a maximum tilt angle,similar to Equation 2. Further, (gamma) is a value indicating tiltaccuracy that can be maintained by a person, for example, without handshaking, similar to Equation 1.

In the example shown in FIG. 13( a), the angle Vmax is assumed to be 180degrees and the angle (gamma) is assumed to be 45 degrees. In this case,the number of the images to be displayed is calculated as M=5((180degrees/45 degrees)+1). Further, the images belonging to the group to bedisplayed are images (C0 to C5) 510 to 515 shown in FIG. 13( b).

Here, in the example shown in FIG. 13, the number of the images (C0 toC5) 510 to 515 (6) is greater than the number M of images to bedisplayed (5). When the number of images belonging to the group to bedisplayed is greater than the number M of images to be displayed (5), itis necessary to perform interleaving so that the number of images to bedisplayed is equal to or less than the number M of images to bedisplayed. In FIG. 13( b), the interleaving process (an extractingprocess) will be described.

In FIG. 13( b), a flow of the interleaving process of images to bedisplayed is shown. In this example, an example in which five images areextracted from the six images (images C0 to C5) when 5 is determined asthe number M of the images to be displayed is shown.

For example, when the manipulation method determination unit 180extracts the images to be displayed, Equation 5 is calculated for eachimage Ci belonging to the group to be displayed.

i mod D<(D−M)  Equation 5

Here, x mod y is a value indicating x divided by y. Further, D is thenumber of images to be displayed and M is the number of images that canbe displayed.

Specifically, the manipulation method determination unit 180 determineswhether i mod D<(D−M) is satisfied for each image Ci belonging to thegroup to be displayed. The manipulation method determination unit 180removes the image Ci for which i mod D<(D−M) is satisfied from among theimages to be displayed. That is, the manipulation method determinationunit 180 extracts the image Ci for which i mod D<(D−M) is not satisfied,as the image to be displayed, and uses the extracted image Ci as animage CCi.

For example, in the example shown in FIG. 13( b), since D=6, M=5, andD−M=1, the image C0 is excluded from the images to be displayed. Thatis, CCi (={C1, C2, C3, C4, C5}={CC0, CC1, CC2, CC3, CC4}) is extractedfrom among the image group (image Ci (0<=i<=5)).

In FIG. 13( c), an example in which the extracted image CCi and the tiltangle (reference angle) Vi are associated with each other isschematically shown. As shown in FIG. 13( c), the extracted CCi={C1, C2,C3, C4, C5} is assigned to the tilt angle (reference angle) Vi, andimage CCi is associated with the tilt angle (reference angle) Vi.

Display Area Setting Example of Indicator

FIG. 14 schematically shows a method of setting the display area of theindicator in the manipulation method determination unit 180 according tothe first embodiment of the present invention. In FIG. 14( a), theimages (C1 to C5) 511 to 515 extracted in the example shown in FIG. 13are shown.

In FIG. 14( b), a relationship between the set display area and theindicator is schematically shown. When the image contents in which theangle information is not recorded in the imaging operation are displayedas shown in FIG. 13, the image contents are assigned to the tilt angles(reference angles) Vi at certain intervals. Accordingly, even when thedisplay area of the indicator is set as shown in FIG. 14( b), the imagecontents may be associated with the tilt angles (reference angles) Viand assigned at certain intervals.

In the foregoing, the example in which the images to be displayed, andthe manipulation support information (e.g., the indicator) indicatingthe total number of images to be displayed and the display imagepositions (positions in a display order) in all the images are displayedwhen the group image display mode has been set has been shown. Here, theimages to be displayed, and manipulation support information (e.g., tiltmeter) indicating the relationship between the posture of the imagingdevice 100 and image advancing or image returning may be displayed.Further, such manipulation support information (e.g., the indicator andthe tilt meter) may be simultaneously displayed. Hereinafter, an examplein which the manipulation support information (tilt meter) therelationship between the posture of the imaging device 100 and imageadvancing or image returning is displayed will be described.

Display Transition Example of Tilt Meter Based on Rotation Angle

FIG. 15 shows a display transition example of manipulation supportinformation (a tilt meter) displayed by the group image reproductionunit 160 in the first embodiment of the present invention. In thisexample, a display transition example when image advancing or imagereturning is performed according to the rotation angle of the imagingdevice 100 is shown. Further, in this example, an example in which thereference angle when image advancing or image returning is performed is20 degrees and image advancing or image returning is performed each timethe tilt angle of the imaging device 100 exceeds 20 degrees is shown. Inthe example shown in FIGS. 15 to 17C, a corresponding angle is indicatedon each rectangle of the tilt meter.

In FIG. 15( a), a transition of the imaging device 100 when the imagingdevice 100 is tilted to the right by the user is shown. In FIG. 15( b),when the imaging device 100 is tilted to the right by the user, adisplay transition of the tilt meter displayed according to the tiltangle is shown. FIGS. 15( a) and (b) correspond to each other, but aportion of FIG. 15( b) is omitted.

For example, a state 611 shown in the top of FIG. 15( a) is a state inwhich the imaging device 100 exists on the same plane as a horizontalplane 330. In this case, as shown in the state 601 in the top of FIG.15( b), only a rectangular area at a middle portion of the tilt meterhas a different display aspect (e.g., black color) from other areas.Hereinafter, the different display aspect from the other areas isreferred to as a black color.

For example, as in states 612 and 613 shown in FIG. 15( a), a case inwhich the user rotates the imaging device 100 to the right in a state inwhich the user holds the imaging device 100 is assumed. In this case,when the rotation angle (theta) at which the imaging device 100 isrotated to the right reaches 5 degrees, the two rectangular areas at theright from the middle of the tilt meter become black as shown in a state602 of FIG. 15( b), but image advancing is not performed. Further, whenthe user rotates the imaging device 100 to the right in a state in whichthe user holds the imaging device 100, the rectangular areas at theright from the middle of the tilt meter become sequentially blackaccording to the rotation angle (theta). Specifically, when the rotationangle (theta) reaches 10 degrees, the two rectangular areas at the rightfrom the middle of the tilt meter become black (state 603), and when therotation angle (theta) reaches 15 degrees, the three rectangular areasat the right from the middle of the tilt meter become black (state 604).Here, in the case of the state 601 to 604, image advancing is notperformed.

Subsequently, when the rotation angle (theta) at which the imagingdevice 100 is rotated to the right reaches 20 degrees, all the portionsat the right from the middle of the tilt meter are black as shown in astate 605, and an indication of reaching the tilt of image advancing isdisplayed and image advancing is performed.

After image advancing is performed and a next image is displayed on theinput/output panel 101, only the rectangular area in the middle portionof the tilt meter becomes black as shown in state 606. That is, the tiltmeter in the state 606 becomes the same as the state in which theimaging device 100 exists on the same plane as the horizontal plane 330(state 601).

Similarly, when the user rotates the imaging device 100 to the rightrotation in a state in which the user holds the imaging device 100, therectangular area in the tilt meter becomes black according to therotation angle (theta), as shown in states 607 and 608 of FIG. 15( b).

Here, when the imaging device 100 is rotated to the right to an angle atwhich the last image among the images to be displayed is sent, the tiltmeter enters a state that is the same as the state 605. Accordingly,even when further rotation to the right is performed, an indicationindicating that image advancing is not performed may be notified of.

Similarly, when the imaging device 100 is rotated to the left, therectangular areas at the left from the middle of the tilt meter becomesequentially black.

While the example in which image advancing or image returning isperformed each time the imaging device 100 exceeds the reference angle(e.g., 20 degrees) has been shown in the foregoing, image advancing orimage returning may be performed by another manipulation method.Hereinafter, an example in which image advancing or image returning isperformed by another manipulation method is shown.

FIG. 16 shows a display transition example of the manipulation supportinformation (the tilt meter) displayed by the group image reproductionunit 160 in the first embodiment of the present invention. In thisexample, a display transition example in which image advancing and imagereturning are sequentially performed when the rotation angle of theimaging device 100 reaches a certain angle and then such a state ismaintained is shown. Further, in this example, an example in which imageadvancing is sequentially performed in the case in which the referenceangle when image advancing or image returning is performed is 20 degreesand the tilt angle of the imaging device 100 is maintained at 20 degreesis shown. In FIGS. 16 to 17C, the transition example of the imagingdevice 100 shown in FIG. 15( a) is omitted.

In FIG. 16( a), a display transition of the tilt meter displayedaccording to the tilt angle when the imaging device 100 is tilted to theright by the user is shown. States 621 to 625 shown in FIG. 16( a)correspond to the states 601 to 605 shown in FIG. 15( b). That is, whenthe user rotates the imaging device 100 to the right in a state in whichthe user holds the imaging device 100 as in the state 612 and 613 of theimaging device 100 shown in FIG. 15( a), the same states as the states601 to 604 shown in FIG. 15( b) are obtained. Further, in the case ofthe states 621 to 624, image advancing is not performed.

Subsequently, when the rotation angle (theta) at which the imagingdevice 100 is rotated to the right reaches 20 degrees, all rightportions from a middle of the tilt meter become black as shown in thestate 625, an indication of reaching the tilt of image advancing isdisplayed, and image advancing is performed.

In FIG. 16( b), a display example of the tilt meter when the rotationangle (theta) at which the imaging device 100 is rotated to the rightreaches 20 degrees and then the rotation angle (theta) is maintained (orwhen a state of the rotation angle (theta) or more is maintained) isshown.

When the rotation angle (theta) at which the imaging device 100 isrotated to the right reaches 20 degrees and then the rotation angle(theta) is maintained, image advancing is performed, and after a nextimage is displayed on the input/output panel 101, the state transitionsto the state 626. This state 626 is a state in which the state 625 ismaintained. That is, the tilt meter in the state 626 becomes the same asthe state (state 625) in which the rotation angle (theta) at which theimaging device 100 is rotated to the right reaches 20 degrees. Further,in the state in which the rotation angle (theta) is maintained, imageadvancing is performed at certain intervals.

In FIG. 16( c), a display example of the tilt meter when the rotationangle (theta) at which the imaging device 100 is rotated to the rightreaches 20 degrees and then the rotation angle (theta) is returned isshown.

When the rotation angle (theta) at which the imaging device 100 isrotated to the right reaches 20 degrees and then the rotation angle(theta) is returned (i.e., when the rotation angle (theta) is less than20 degrees), image advancing is performed, a next image is displayed onthe input/output panel 101, and the state transitions to state 627. Thatis, when the rotation angle (theta) at which the imaging device 100 isrotated to the right reaches 20 degrees and then the rotation angle(theta) is returned, the display of the tilt meter is returned accordingto the returned angle and image advancing is ceased.

Similarly, when the imaging device 100 is rotated to the left, therectangular areas at the left from the middle of the tilt meter becomesequentially black.

In the foregoing, an example in which image advancing or image returningis performed based on whether the rotation angle of the imaging device100 exceeds a reference angle of image advancing or image returning hasbeen shown. Here, for example, image advancing or image returning may beperformed based on whether the rotational angular velocity of theimaging device 100 exceeds a reference. Hereinafter, an example in whichimage advancing or image returning is performed based on whether therotational angular velocity of the imaging device 100 exceeds areference will be described.

Display Transition Example of Tilt Meter Based on Rotational AngularVelocity

FIGS. 17A to 17C shows a display transition example of manipulationsupport information (a tilt meter) displayed by the group imagereproduction unit 160 in the first embodiment of the present invention.In this example, a display transition example when image advancing orimage returning is performed according to the rotational angularvelocity (rotation angle/sec) of the imaging device 100 is shown. Thatis, in this example, an example in which image advancing or imagereturning is performed according to whether the device reaches a certainangular velocity by one rotation manipulation is shown. Further, in thisexample, an example in which the rotational angular velocity (rotationangle/sec) when image advancing or image returning is performed is 20degrees/sec and image advancing or image returning is performed eachtime the rotational angular velocity of the imaging device 100 exceeds20 degrees/sec is shown.

In FIG. 17A, a display transition of the tilt meter displayed accordingto a rotational angular velocity when the imaging device 100 is rotatedat 5 degrees/sec to the right by the user is shown (states 631 and 632).In this case, since the rotational angular velocity does not reach 20degrees/sec, image advancing is not performed and the original state isreturned (i.e., the state is returned from the state 632 to the state631).

In FIG. 17B, a display transition of the tilt meter displayed accordingto a rotational angular velocity when the imaging device 100 is rotatedat 10 degrees/sec to the right by the user is shown (states 633 and634). In this case, since the rotational angular velocity does not reach20 degrees/sec, image advancing is not performed and the original stateis returned (i.e., the state is returned from the state 634 to the state633). Similarly, when the imaging device 100 is rotated at 15degrees/sec to the right by the user, the display state of the tiltmeter is changed according to the tilt angular velocity, but since therotational angular velocity does not reach 20 degrees/sec, imageadvancing is not performed. Similarly, even in this case, the originalstate is returned.

In FIG. 17C, a display transition of the tilt meter displayed accordingto a rotational angular velocity when the imaging device 100 is rotatedat 20 degrees/sec to the right by the user is shown. In this case, sincethe rotational angular velocity reaches 20 degrees/sec, image advancingis performed. When the image advancing is performed and then the stateof such an angle is maintained (i.e., in the case of 0 degree/sec), theoriginal state is returned (i.e., the state is returned from the state636 to the state 637). Further, when the rotational angular velocity ismaintained (or when a state of the rotational angular velocity or moreis maintained), the display state of the tilt meter is maintainedaccording to a maintained period and image advancing is sequentiallyperformed.

In FIGS. 17B and 17C, some of a display transition from the state 633 tothe state 634 and a display transition from the state 635 to the state636 are omitted.

Similarly, when the imaging device 100 is rotated to the left, therectangular areas at the left from the middle of the tilt meter becomesequentially black.

Display Example of Manipulation Support Information

FIGS. 18 and 19 schematically show a relationship between a displaytransition in the input/output panel 101 and the posture of the imagingdevice 100 in the first embodiment of the present invention.

In FIG. 18, a transition example when the manipulation supportinformation (the indicator) superimposed on the image to be displayed isdisplayed on the input/output panel 101 is shown. Further, in FIG. 19, atransition example in which manipulation support information (tiltmeter) superimposed on the image to be displayed is displayed on theinput/output panel 101 is shown. Further, in FIGS. 18( b) and 19(b), atransition example of the posture of the imaging device 100 when animage is displayed on the input/output panel 101 is shown. In theexamples shown in FIGS. 18( b) and 19(b), a transition example when theposture of the imaging device 100 upon setting of the group imagedisplay mode is located on the plane of a horizontal plane 330 is shown.Further, in FIGS. 18 and 19, images displayed on the input/output panel101 shown in each of in FIGS. 18( a) and 19(a) and postures of theimaging device 100 shown in each of in FIGS. 18( b) and 19(b) are shownto show a corresponding relationship at the left and right.

In FIGS. 18( a) and 19(a), the transition of the image displayed on theinput/output panel 101 is shown. The images 651 to 653 displayed on theinput/output panel 101 shown in FIGS. 18( a) and 19(a) are imagesobtained by imaging a face of the same person from different views.Accordingly, the user can obtain the feeling of viewing athree-dimensional object (a face of a person) by rapidly performing usermanipulation to tilt the imaging device 100 to the left and right andrapidly performing image advancing or image returning. That is, the usermanipulation to tilt the imaging device 100 to the left and rightenables the user to obtain the feeling of viewing apseudo-three-dimensional image. Further, the group to which the images651 to 653 belong includes seven images (including the image 651 to653). The image 651 is a representative image of the group and theimages 652 and 653 are first and last images in a display order.

First, the case in which the manipulation support information (theindicator) superimposed on the image to be displayed is displayed on theinput/output panel 101 will be described with reference to FIG. 18. Asdescribed above, the representative image (image 651) is displayed onthe input/output panel 101 directly after the group image display modeis set. In this case, the manipulation support information 403superimposed on the image 651 is displayed, similar to FIG. 7 C. Here,since the manipulation support information 403 is deleted or reducedwhen image advancing or image returning is performed for a certain time(or a certain number of times) by the user manipulation, the displayexample is omitted in FIGS. 18( a) and 19(b).

Here, a case in which the image 652 is displayed on the input/outputpanel 101 when the group image display mode has been set is assumed. Inthis case, since the image 652 is a first image in a display order amongthe seven images in the same group, a rectangular area at a left end inthe indicator 660 has a different display aspect from other rectangularareas. In this case, it is difficult to perform further image returning.

Further, a case in which an image 653 is displayed on the input/outputpanel 101 when the group image display mode has been set is assumed. Inthis case, since the image 653 is a last image in the display orderamong the seven images in the same group, a rectangular area at a rightend in the indicator 660 has a different display aspect from the otherrectangular areas. In this case, it is difficult to perform furtherimage advancing.

Thus, a mark indicating order relating to the images displayed on theinput/output panel 101 in all of the plurality of images (e.g., theorder specified by the normalization angle information) can be displayedas the indicator.

It is possible to easily recognize positions in a display order ofimages to be displayed in the group by displaying the images to bedisplayed and the indicator 660 when the group image display mode isset. Accordingly, it is possible to easily perform the image advancingor image returning manipulation while viewing the images to be displayedwhen performing the image advancing or image returning manipulation.

Next, a case in which the manipulation support information (the tiltmeter) superimposed on the image to be displayed is displayed on theinput/output panel 101 will be described with reference to FIG. 19.

Here, a case in which an image 652 is displayed on the input/outputpanel 101 when the group image display mode has been set is assumed. Inthis case, since the image 652 is a first image in the display orderamong the seven images in the same group, the rectangular areas from amiddle of the tilt meter 670 to a left end have a different displayaspect from the other rectangular areas. In this case, it is difficultto perform further image returning.

Further, a case in which an image 653 is displayed on the input/outputpanel 101 when the group image display mode has been set is assumed. Inthis case, since the image 653 is a last image in the display orderamong the seven images in the same group, the rectangular areas from themiddle of the tilt meter 670 to a right end have a different displayaspect from the other rectangular areas. In this case, it is difficultto perform further image advancing.

Thus, it is possible to display the mark indicating the change amount ofthe posture of the imaging device 100 necessary for changing the displaystate in the input/output panel 101, as the tilt meter. That is, it ispossible to display the mark indicating the change amount of the postureof the imaging device 100 necessary for performing image advancing orimage returning on a plurality of images, as the tilt meter.

Since the transition of the display state of the tilt meter 670according to the tilt angle of the imaging device 100 is the same as thetransition examples of FIGS. 15 to 17C, a description thereof will beomitted herein.

Thus, the user can easily recognize the tilt to display each image to bedisplayed by displaying the images to be displayed and the tilt meter670 when the group image display mode has been set. Further, the usercan easily recognize the positions in the display order of the images tobe displayed in the group. Accordingly, the user can easily perform theimage advancing or image returning manipulation while viewing the imagesto be displayed when performing the image advancing or image returningmanipulation.

Operation Example of Imaging Device

FIG. 20 and FIG. 21 are flowcharts showing an example of a processingprocedure of an image content reproduction process in the imaging device100 according to the first embodiment of the present invention.

First, a determination is made as to whether an image content displayinstruction manipulation is performed (step S901), and when the displayinstruction manipulation is not performed, monitoring is continued. Onthe other hand, when the display instruction manipulation is performed(step S901), the control unit 140 sets the representative image displaymode and the representative image reproduction unit 150 displays arepresentative image and non-grouped images on the display unit 111(step S902).

Subsequently, a determination is made as to whether the image advancingmanipulation or image returning manipulation is performed in a state inwhich the representative image display mode has been set (step S903).When the image advancing manipulation or the image returningmanipulation is performed (step S903), the representative imagereproduction unit 150 performs display switching of the image displayedon the display unit 111 (step S904). That is, image advancing or imagereturning of the image displayed on the display unit 111 is performed.

When the image advancing manipulation or image returning manipulation isnot performed (step S903), a determination is made as to whether thegroup image display mode setting manipulation is performed (step S905),and when the group image display mode setting manipulation is notperformed, the process proceeds to step S916. On the other hand, whenthe group image display mode setting manipulation is performed (stepS905), a group image display mode setting process is performed (stepS920). The group image display mode setting process will be explained indetail with reference to FIG. 22.

Subsequently, the group image reproduction unit 160 displays initialmanipulation support information (e.g., the manipulation supportinformation 403 shown in FIG. 7( c)) superimposed on the image on thedisplay unit 111 (step S906). That is, the group image reproduction unit160 decodes the image content acquired from the image content storageunit 200, and renders a display image in the rendering memory 170 basedon the decoded image content. The group image reproduction unit 160simultaneously displays one (representative image) of the imagesrendered in the rendering memory 170 and the initial manipulationsupport information on the display unit 111.

Subsequently, a determination is made as to whether the initialmanipulation support information is displayed (step S907), and when theinitial manipulation support information is not displayed, the processproceeds to step S910. On the other hand, when the initial manipulationsupport information is displayed (step S907), a determination is made asto whether a certain time elapses after the group image display mode isset (step S908). When a certain time elapses after the group imagedisplay mode is set (step S908), the group image reproduction unit 160deletes the initial manipulation support information displayed on thedisplay unit 111 (step S909). On the other hand, when a certain time hasnot elapsed after the group image display mode is set (step S908), adetermination is made as to whether a tilt manipulation by the user isperformed (step S910). That is, the group image reproduction unit 160determines whether the posture of the imaging device 100 is changedabove a certain amount based on the analysis information output from theanalysis unit 130.

When the tilt manipulation by the user is performed (step S910), adetermination is made as to whether the initial manipulation supportinformation is displayed (step S911), and when the initial manipulationsupport information is not displayed, the process proceeds to step S913.On the other hand, when the initial manipulation support information isdisplayed (step S911), the group image reproduction unit 160 deletes theinitial manipulation support information displayed on the display unit111 (step S912).

Subsequently, the group image reproduction unit 160 changes the displaystate of the manipulation support information (the indicator or the tiltmeter) based on the manipulation amount according to the tiltmanipulation by the user and the display image (the image displayed onthe display unit 111) (step S913). Subsequently, when the manipulationamount according to the tilt manipulation by the user exceeds a displayswitching reference, the group image reproduction unit 160 performsdisplay switching (image advancing or image returning) on the imagedisplayed on the display unit 111 (step S914), and the process returnsto step S907. When the manipulation amount according to the tiltmanipulation by the user does not exceed the display switchingreference, the group image reproduction unit 160 does not performdisplay switching (image advancing or image returning) on the imagedisplayed on the display unit 111 and the process returns to step S907.

When a tilt manipulation by the user is not performed (step S910), adetermination is made as to whether the representative image displaymode setting manipulation is performed (step S915), and when the settingmanipulation is performed, the process returns to step S902. On theother hand, when the representative image display mode settingmanipulation is not performed (step S915), a determination is made as towhether the image content display termination manipulation is performed(step S916), and when the display termination manipulation is performed,the image content reproduction process operation is terminated. When theimage content display termination manipulation is not performed (stepS916), a determination is made as to whether the group image displaymode has been set (step S917). When the group image display mode hasbeen set (step S917), the process returns to step S907, and when thegroup image display mode has not been set (that is, the representativeimage display mode has been set), the process returns to step S903.Steps S906 to S920 are one example of a control procedure defined in theclaims. Step S910 is one example of a detection procedure defined in theclaims.

FIG. 22 is a flowchart showing an example of a group image display modesetting process (a processing procedure in step S920 shown in FIG. 20)in a processing procedure of the image content reproduction process inthe imaging device 100 according to the first embodiment of the presentinvention.

First, the group image reproduction unit 160 acquires each image contentbelonging to a group corresponding to the representative image displayedon the display unit 111 upon a group image display mode settingmanipulation from the image content storage unit 200 (step S921).Further, the group image reproduction unit 160 acquires image managementinformation associated with each image content belonging to the groupfrom the image management information storage unit 210 (step S921).

Subsequently, the manipulation method determination unit 180 determineswhether angle information is included in the acquired image managementinformation (step S922). When the angle information is included in theacquired image management information (step S922), the manipulationmethod determination unit 180 normalizes the angle information includedin the acquired image management information for each image content(step S923).

Subsequently, the manipulation method determination unit 180 associatesthe acquired image content with the tilt angle (the reference angle)based on the normalized angle information (normalization angleinformation) (step S924). In this case, when the number of the acquiredimage contents is greater than the number of tilt angles (referenceangles), the manipulation method determination unit 180 performs aninterleaving process on the acquired image contents so that the numberof image contents is in a range of the number of tilt angles (referenceangles). Subsequently, the manipulation method determination unit 180calculates the display area of the indicator based on a correspondencerelationship resulting from the association (step S925).

Further, when the angle information is not included in the acquiredimage management information (step S922), the manipulation methoddetermination unit 180 determines whether the number of the acquiredimage contents is greater than the number of images that can bedisplayed (the number of tilt angles (reference angles)) (step S926).When the number of acquired image contents is equal to or less than thenumber of images that can be displayed (step S926), the process proceedsto step S928. On the other hand, when the number of the acquired imagecontents is greater than the number of images that can be displayed(step S926), the manipulation method determination unit 180 performs aninterleaving process on the acquired image contents so that the numberof images is in a range of the number of images that can be displayed(step S927).

Subsequently, the manipulation method determination unit 180 associatesthe acquired image contents (or the image contents after theinterleaving process) with the tilt angles (reference angles) (stepS928). The manipulation method determination unit 180 then calculatesthe display area of the indicator based on a correspondence relationshipresulting from the association (step S929).

2. Variant

In the foregoing, the example in which the manipulation supportinformation (the tilt meter or the indicator) structured by a rod-shapedrectangle is displayed as the manipulation support information has beenshown. In the variant, an example of the manipulation supportinformation (the tilt meter or the indicator) displayed with anotherstructure is shown. The configuration of the imaging device 100 is thesame as the first embodiment of the present invention except that thedisplay aspect of the manipulation support information differs.Accordingly, hereinafter, a difference with the first embodiment of thepresent invention will be mainly described and a description of the sameportions will be omitted.

Display Example of Tilt Meter

FIGS. 23A and 23B shows manipulation support information (a tilt meter)displayed by the group image reproduction unit 160 and a displaytransition example in the input/output panel 101 in the variant of thefirst embodiment of the present invention.

In FIG. 23A, one example of the manipulation support information (thetilt meter) structured in a circular shape is shown. Correspondingangles are indicated at outer sides of the circle constituting the tiltmeter. Further, the tilt angle of the imaging device 100 is indicated byan arrow from a center of the circle constituting the tilt meter to theposition on the circle corresponding to the tilt angle.

In FIG. 23B, a transition of the image displayed on the input/outputpanel 101 is shown. As described above, the representative image (image701) is displayed on the input/output panel 101 directly after the groupimage display mode is set. In this case, manipulation supportinformation 711 superimposed on the image 701 is displayed, similar toFIG. 7 C. Here, the manipulation support information 711 may be deletedor reduced when image advancing or image returning is performed for acertain time (or a certain number of times) by the user manipulation.

Further, in the example shown in FIG. 23B, a display transition examplewhen image advancing or image returning is performed according to therotation angle of the imaging device 100 is shown. Further, in thisexample, an example in which the reference angle when image advancing orimage returning is performed is 30 degrees and image advancing or imagereturning is performed each time the tilt angle of the imaging device100 exceeds 30 degrees is shown.

For example, a case in which the user rotates the imaging device 100 tothe left in a state in which the user holds the imaging device 100 isassumed. For example, when an angle at which the imaging device 100 isrotated to the left reaches 20 degrees, the tilt meter 712 with an arrowfrom the center of the circle constituting the tilt meter to a positionon the circle corresponding to the rotation angle (20 degrees) isdisplayed on the image 701. Here, in this case, image advancing is notperformed. Further, a case in which the user rotates the imaging device100 to the left in a state in which the user holds the imaging device100 and the rotation angle reaches 30 degrees is assumed. In this case,the tilt meter with an arrow from the center of the circle constitutingthe tilt meter to a position on the circle (a bottom of the circle)corresponding to the rotation angle (30 degrees) is displayed on theimage 701, an indication of reaching a tilt of image advancing isdisplayed, and image advancing is performed.

Similarly, when the imaging device 100 is rotated to the right, a tiltmeter 713 with an arrow from the center of the circle constituting thetilt meter to the position on the circle corresponding to the rotationangle is superimposed on the image 701 and displayed.

FIGS. 24A and 24B shows an example of manipulation support information(a tilt meter) displayed by the group image reproduction unit 160 and adisplay transition in the input/output panel 101 in the variant of thefirst embodiment of the present invention.

In FIG. 24A, an example of manipulation support information (a tiltmeter) structured in a circular shape is shown. Corresponding rotationalangular velocities are indicated at outer sides of the circleconstituting the tilt meter. Further, the rotational angular velocity ofthe imaging device 100 is indicated by an arrow from the center of thecircle constituting the tilt meter to a position on the circlecorresponding to the rotational angular velocity.

In FIG. 24B, a transition of the image displayed on the input/outputpanel 101 is shown. This example is the same as that shown in FIG. 23Bin that a representative image (an image 701) is displayed on theinput/output panel 101 and manipulation support information 711 issuperimposed on the image 701 and displayed.

Further, in the example shown in FIG. 24B, a display transition examplewhen image advancing or image returning is performed according to therotational angular velocity of the imaging device 100 is shown. Further,in this example, an example in which the rotational angular velocity(rotation angle/sec) when image advancing or image returning isperformed is 20 degrees/sec and image advancing or image returning isperformed each time the rotational angular velocity of the imagingdevice 100 reaches 20 degrees/sec is shown.

For example, a case in which the user rotates the imaging device 100 tothe left in a state in which the user holds the imaging device 100 andthe rotational angular velocity reaches 20 degrees/sec is assumed. Inthis case, a tilt meter 721 with an arrow from the center of the circleconstituting the tilt meter to a position on the circle corresponding tothe rotational angular velocity (20 degrees/sec) is displayed on theimage 701, an indication of reaching a rotational angular velocity ofimage advancing is displayed, and image advancing is performed. Theimage 702 is displayed by image advancing. Further, when a state of theangle is maintained after image advancing (i.e., in the case of 0degree/sec), the original state is returned. That is, a tilt meter 722with an arrow from the center of the circle constituting the tilt meterto a position on the circle (a top of the circle) corresponding to therotational angular velocity (0 degree/sec) is displayed on the image702.

Similarly, when the imaging device 100 is rotated to the right, a tiltmeter with an arrow from the center of the circle constituting the tiltmeter to the position on the circle corresponding to the rotationalangular velocity is displayed.

In the tilt meter shown in FIGS. 23A to 24B, a range of the top of thecircle and a position on the circle indicated by the arrow (a range of apan system specified by the two points and the center of the circle) mayhave a different display aspect from other areas to be easily recognizedby the user.

Display Example of Indicator

FIGS. 25 to 28 show a display transition example in the input/outputpanel 101 in the variant of the first embodiment of the presentinvention. In such an example, a variant of a display aspect ofmanipulation support information (indicator) displayed by the groupimage reproduction unit 160 is shown. Further, in FIGS. 25 and 27, anexample in which display support information (e.g., icon) 802 and 812indicating the type of a group to be displayed is displayed with theindicator is shown.

In FIGS. 25 and 26, a display transition example when a multi-view imageis displayed is shown. In FIG. 25, an example in which an indicator 801and display support information 802 indicating that the type of a groupto be displayed is “multi-view image” are simultaneously displayed isshown. In FIG. 26, an example in which an indicator 811 indicating thatthe type of a group to be displayed is “multi-view image” is displayedis shown.

In FIGS. 27 and 28, a display transition example when a multi-shot imageis displayed is shown. In FIG. 27, an example in which the indicator 821and display support information 822 indicating that the type of thegroup to be displayed is a “multi-shot image” are simultaneouslydisplayed is shown. In FIG. 28, an example in which an indicator 831indicating that the type of the group to be displayed is a “multi-shotimage” is displayed is shown.

Since a display state in the indicator shown in FIGS. 25 to 28 is thesame as the example shown in FIG. 18, a description thereof will beomitted herein.

Thus, based on the correlativity (e.g., corresponding to the group type)between the images in the image group associated with the imagedisplayed on the input/output panel 101, a display aspect of theindicator is changed from the image group to the image group and thenthe indicator is displayed.

Further, the user can easily perform a manipulation to display eachimage to be displayed by displaying the image to be displayed and thedisplay support information indicating the type of the group to bedisplayed when the group image display mode has been set. That is, whenimage advancing or image returning manipulation is performed, the usercan easily perform the manipulation while viewing the image to bedisplayed.

As described above, in the embodiment of the present invention, theinitial manipulation support information (e.g., the manipulation supportinformation 403 shown in FIG. 7( c)) is displayed upon setting of thegroup image display mode. Accordingly, it is possible to prevent adesired image from missing due to, for example, a bad way of holding theimaging device 100 and switching of a displayed view upon initiation ofimage reading.

Further, for example, the user can easily recognize a tilt angle atwhich a view is switched by displaying the tilt meter when themulti-view image is displayed upon setting of the group image displaymode. Further, for example, even when an image in which a change of aswitched view is small is displayed, the user can easily recognize thatthe image is switched by the user performing a manipulation to tilt thedevice.

Further, for example, the user can easily recognize one of all views towhich a currently displayed view corresponds by displaying the indicatorwhen the multi-view image is displayed upon setting of the group imagedisplay mode. Further, for example, when an image of a view photographedfrom a rightmost side is displayed, the user can easily recognize thatthe view is not switched even though a right side is further tilteddown.

In the example shown above, the example in which the display state inthe input/output panel 101 is changed by rotating the imaging device 100about the vertical direction of the imaging device 100 has been shown.Here, the embodiment of the present invention may be applied to a casein which the display state in the input/output panel 101 is changed byrotating the imaging device 100 about the horizontal direction of theimaging device 100. In this case, for example, the image may be changedby up and down tilt manipulations, and the indicator or the tilt metermay be vertically arranged and displayed. Further, for example, when animage in which there is a view in both directions of up and down andleft and right is displayed, view switching may be performed by a tiltmanipulation in both directions of up and down and left and right, andthe indicator or the tilt meter may be displayed for both directions ofup and down and left and right. The embodiment of the present inventionmay also be applied to a case in which the display state in theinput/output panel 101 is changed by sliding (parallel movement) towardany of up, down, left, right, backward and forward directions of theimaging device 100.

While the imaging device has been described by way of example in theembodiments of the present invention, the embodiments of the presentinvention may be applied to an image processing device capable ofdisplaying image contents stored in the recording medium on the displayunit. For example, the embodiments of the present invention may beapplied to image processing devices, such as a mobile phone, anavigation system, and a portable media player with an imaging function.

The embodiment of the present invention illustrates one example forembodying the present invention, and the matters in the embodiment ofthe present invention and the specified matters of the invention in theclaims have a correspondence relationship, as described in theembodiment of the present invention. Similarly, the specified matters ofthe invention in the claims and the matters in the embodiment of thepresent invention having the same names as the specified matters have acorrespondence relationship. Here, the present invention is not limitedto the embodiments, and various variations may be made to theembodiments without departing from the spirit and scope of the presentinvention.

Further, the processing procedure described in the embodiments of thepresent invention may be regarded as a method including a series ofprocedures or as a program for causing a computer to execute a series ofprocedures or a recording medium having the program stored thereon. Therecording medium may be, for example, a compact disc (CD), a mini disc(MD), a digital versatile disc (DVD), a memory card, or a Blu-ray disc(registered trademark).

REFERENCE SIGNS LIST

-   100 Imaging device-   101 Input/output panel-   102 Shutter button-   110 Input/output unit-   111 Display unit-   112 Manipulation receiving unit-   120 Posture detection unit-   130 Analysis unit-   140 Control unit-   150 Representative image reproduction unit-   160 Group image reproduction unit-   170 Rendering memory-   180 Manipulation method determination unit-   200 Image content storage unit-   210 Image management information storage unit

1. A method of a processor to control a display of images comprising:detecting posture of an image display device with a sensor; sequentiallydisplaying images of a group of images on the image display device basedon changes in the detected posture; and displaying a posture indicatoron the image display device, the indicator representing a relationbetween a change in the detected posture and an image of the group ofimages.
 2. The method of claim 1 wherein the indicator comprises agraphic tilt meter.
 3. The method of claim 1 wherein the processorassociates a normalized tilt angle with an image of the sequentiallydisplayed images.
 4. The method of claim 3 wherein the normalized tiltangle comprises an angle determined as a function of image capture angleinformation and a maximum tilt angle.
 5. The method of claim 4 whereinthe image capture angle information comprises a range of captured imageangles.
 6. The method of claim 5 wherein the maximum tilt anglecomprises a display viewing angle limit.
 7. The method of claim 1further comprising sequentially displaying entrance images for groups ofimages on the image display device based on changes in the detectedposture.
 8. The method of claim 1 wherein the sensor comprises agyroscopic sensor.
 9. An apparatus for control of displaying of imagescomprising: an image display; a sensor to detect posture of the imagedisplay; a processor to control sequentially displaying images of agroup of images on the image display based on changes in the detectedposture; and wherein the processor controls a display of a postureindicator on the image display, the indicator representing a relationbetween a change in the detected posture and an image of the group ofimages.
 10. The apparatus of claim 9 wherein the indicator comprises atilt meter.
 11. The apparatus of claim 9 wherein the processor controlsassociating a normalized tilt angle with an image of the sequentiallydisplayed images.
 12. The apparatus of claim 9 wherein the detection ofposture comprises a detection of angular velocity and wherein theprocessor controls changing the display based on the angular velocity.13. The apparatus of claim 11 wherein the processor controls calculatingthe normalized tilt angle as a function of image capture angleinformation and a maximum tilt angle.
 14. The apparatus of claim 13wherein the image capture angle information comprises a range ofcaptured image angles.
 15. The apparatus of claim 13 wherein the maximumtilt angle comprises a display viewing angle limit.
 16. The apparatus ofclaim 9 wherein the processor controls sequentially displaying entranceimages for groups of images on the image display device based on changesin the detected posture.
 17. The apparatus of claim 1 wherein the sensorcomprises a gyroscopic sensor.
 18. A system for control of displayingimages comprising: a display means for image displaying; a posture meansfor sensing posture of the means for image displaying; a control meansfor controlling sequentially displaying images of a group of images onthe display means based on changes in the sensed posture; and whereinthe control means controls a display of a means for posture indicatingon the image display, the means for posture indicating representing arelation between a change in the sensed posture and an image of thegroup of images.
 19. The system of claim 18 wherein the control meanscontrols associating of a normalized tilt angle with an image of thesequentially displayed images.
 20. The system of claim 19 wherein thecontrol means controls calculating the normalized tilt angle as afunction of image capture angle information and a maximum tilt angle.21. The system of claim 20 wherein the image capture angle informationcomprises a range of captured image angles determined by the posturemeans.